Validation of infant immunization billing codes in administrative data

Ontario has a single payer provincial health insurance program. Administrative data may provide a potentially robust source of information for post-marketing vaccine studies. Vaccine-specific immunization billing codes were introduced in 2011. Our objective was to validate Ontario's universal health care administrative datasets to assess infant immunization status. Electronic medical record data from the Electronic Medical Record Administrative data Linked Database (EMRALD) was used as the reference standard to calculate performance characteristics of the Ontario Health Insurance Plan (OHIP) database vaccine-specific and general immunization codes for 4 primary infant immunizations: diphtheria, tetanus, acellular pertussis, inactivated polio, Haemophilus influenzae type B (DTaP-IPV-Hib) combination vaccine, pneumococcal conjugate vaccine, measles, mumps, rubella (MMR) vaccine, and meningococcal conjugate serogroup C vaccine. OHIP billing claims had specificity ranging from 81% to 92%, sensitivity 70% to 83%, positive predictive value (PPV) 97% to 99%, and negative predictive value (NPV) 13% to 46% for identifying the various specific vaccines in administrative data. For cohorts vaccinated in the new code introduction phase, using both the vaccine-specific and general codes had higher sensitivity than the vaccine-specific codes alone. In conclusion, immunization billing claims from administrative data in Ontario had high specificity and PPV, moderate sensitivity, and low NPV. This study identifies some of the applications of utilizing administrative data for post-marketing vaccine studies. However, limitations of these data decrease their utility for measuring vaccine coverage and effectiveness. Therefore, the establishment of a comprehensive and linkable immunization registry should be a provincial priority.